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FOREWORD 



This manual has been designed for use 
by teachers engaged in research for 
personal or professional reasons, for 
school principals interested in evaluating 
various aspects of the curriculum of 
their schools and for administrators, 
supervisors, and other school personnel 
interested in becoming involved in the 
research process. It may also be of value 
to individuals within a government 
agency or private industry interested in 
planning research with Adult Basic 
Education Programs that they arj using or 
intend to use. 

It has been written to serve both 
as a refresher for those familiar with 
educational research and as a guide for 
the neophyte researcher. 



INTRODUCTION 



/ 




ERIC 



A major development in the area of adult education during the 
past four years has been the growth of basic education classes 
throughout the United States. Motivated by the awareness that 25 
million adult citizens of the United States are functionally illiterate, 
unable to read or write above the eighth grade level, the Congress, 
early in 1964, provided funds to expand adult basic education pro- 
grams in local communities under the Economic Opportunity Act 
Title II-B. 

As these programs evolved, adult educators realized the need for 
a communication skills curriculum geared specifically for adults. 
They found that due to the unique characteristics of the under- 
educated adult, the traditional elementary materials devised for 
youngsters were not adequate. This problem was magnified by the 
awareness that many of the functionally illiterate adults come from 
deprived backgrounds and have environmental pressures that are 
oppressive and that adversely affect tneir learning. Educators of 
secondary school students also had a need for programs to increase 
competency and fluency in reading and the allied communication 
skills, especially for the student from a disadvantaged environment 
or the potential dropout. 

Various materials were developed by publishers to fill these cur- 
riculum needs. State or city agencies dealing with problems of basic 
education, federally funded projects attempting to raise the levels 
of the disadvantaged, private industries now involved with basic 



and technical education for their employees, or secondary schools 
faced with the need to reinterest discouraged and frustrated teen- 
agers or young adults who are potential dropouts are faced with a 
need to determine the effectiveness of these newly available edu- 
cational programs. 

To measure the success of a program, educators need to de- 
termine the changes they want to bring a K out in their students 
during a specific period of time, and these changes then become 
the objectives of the curriculum. The changes might include mastery 
of content, usually achieved in connection with the study of a 
particular subject, and the reactions of students to this content, 
including ways of thinking or acquisition of skills. 

In addition, educators often wish to bring about changes in the 
affective domain. These changes include improving the self-concept, 
developing more resourcefulness, encouraging independence, raising 
levels of aspiration, and developing social living skills. 

Once a set of objectives has been selected, evaluation is neces- 
sary to provide relevant information for curricular decisions and the 
means of sharing insights with others. It is an ongoing process of 
securing information to aid educators in deciding how well they are 
meeting their educational objectives and whether or not it is neces- 
sary to augment or to alter their curricula to meet their objectives. 
Intelligent decision making requires the selection of evaluation 
methods that will provide this necessary information. 

Methods of evaluation include any means of securing valid evi- 
dence of attainment of objectives, such as paper-and-pencil tests, 
observations of behavior, and performance records. Evaluation, 
however, involves more than the methods themselves. It includes 
(1] clarification of objectives to the extent of describing which be- 
haviors represent attainment in a particular area; (2) the develop- 
ment and use of various procedures for obtaining evidence of 
changes in students; (3) appropriate ways of summarizing and inter- 
preting that evidence; and (4) the use of information gained to 
improve curriculum, teaching, and guidance. 

The nature of an evaluation program depends on how each ob- 
jective Is defined and pursued, It also depends on the purposes for 
which the results of the evaluation are used. A school or educa- 
tional program concerned only with mastery of information or 
attainment of a certain level of proficiency will confine its efforts 
to assessing that achievement in terms of local or national norms. 
If, on the other hand, the objectives include development of various 
attitudes and social behaviors, a broader range of evidence and 
special ways of appraising that evidence other than comparisons of 
test scores is needed, Evaluation conceived of in this manner is an 
Integral part of curriculum Innovation beginning with determination 
of objectives ar *. ending wilh assessment of their attainment. 
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Caution needs to be exercised with any method of evaluation, 
however, for the information gained is only as valid vnd reliable 
as the procedures used in evaluation. Of particular importance in 
evaluation is the need to examine an instructional innovation in 
relation to a particular educator’s objectives which are based upon 
the local conditions and needs. For this reason it is urged that the 
innovations be evaluated in the instructional settings in which they 
are being used. 

The use of evaluation as a means of answering questions about 
the relative benefits of different curricula is of infinite value to the 
evaluator, the administrator, the teachers, the students and the 
community. When questions concerning benefit to students can be 
answered by research methods, valid decisions can be made about 
curriculum innovations. 

PURPOSE 

The purpose of this manual is tc aid an educator in evaluating 
the Learning 100 ( L-100) system within his instructional setting. 
The manual was designed to famiFarize the prospective evaluator 
with the procedures necessary for evaluation, including planning 
for evaluation, stating objectives in operational terms, preparing a 
research design, administering the study, analyzing and interpreting 
the data acquired, reporting the results, and translating the findings 
into curriculum improvements. 

METHODOLOGY 

In order to fulfill the purpose for which this manual was de- 
veloped, the information contained herein has been arranged in the 
following manner: 

Chapter 2 is a brief description of the Learning 100 system. For 
a more comprehensive understanding of the purpose of the L-100 
system and its implementation, it is recommended that the L-100 
Instructor’s Manual be studied and desirably one should visit a 
program in operation to observe the interaction between the stu- 
dents and the materials. 

Chapter 3 deals with the principles cf designing evaluation and 
centers first on the procedures to be followed in determining a 
research problem. The importance of reviewing related literature 
and asking relevant questions is emphasized. It is suggested that 
the educator’s specific instructional objectives be examined In rela- 
tion to tht overarching and subsystem objectives of the L-100 sys- 
tem. (Overarching objectives are general, broad, and longitudinal in 
nature. Subsystem objectives, on the other hand, are more specific 
in nature and provide the plan of action for carrying out these 




overarching objectives.) Chapter 3 also discusses methods for de- 
termining hypotheses to be tested, selecting research designs, select- 
ing sample groups from a popula ion, determining methods for 
collecting data to evaluate objectives, determining methods of 
analyzing data, preparing flow charts, administering the study, 
analyzing and interpreting the data, reporting the results, and im- 
plementing the research findings in the curricu^m. 

Appendix A is a listing of the overarching and subsystem ob- 
jectives of the L-100 system and Appendix B includes a taxonomy 
of procedures for selection of data collection methods and instru- 
ments. A bn'jf description of statistical tests appropriate for class- 
room or school district research which may serve as a guide for 
analyzing data is offered in Appeidix C. Appendix D suggests 
sample questionnaire items which n ight be included in an evalua- 
tion study. To assist the person ujing the manual, a glossary of 
research and evaluation terms follov/s. A selected bibliography has 
been included. Finally, a graphic guide is included to aid the evalu- 
ator in relating the contents of this manual to his particular in- 
structional situation. 
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PURPOSE 

Learning 100 is a multimedia, multimodal, multilevel communica- 
t ion skills system specifically designed for use by the uneducated 
^pr undereducated young person or adult who is unable to function 
Effectively in school or in the working world because he lacks 
basic literacy end is deficient in communication skills. This total 
systems approach to basic education combines audiovisual and in- 
strument techniques with a variety of printed materials in inter- 
dependent, interlocking cycles of instruction designed to facilitate 
the accomplishment of specific learning in the areas of reading, 
writing, listening, speaking, observing, and in the thinking skills 
which underlie these acts. 

Learning 100 was designed to help the potential or actual school 
dropout, the working illiterate adult whose employability is threat- 
ened by lack of competency in reading and in listening, and the 
poverty-plagued adult who has experienced long periods of jobless' 
ness and welfare relief. For many of the target population the 
motivation to remain in school or return to school has been non- 
existent. For these individuals, traditional schooling has been asso- 
ciated with frustration and failure, and as a result, many have 
developed deep-seated hostilities to formal learning situations. For 
these persons, Learning 100 provides dramatically new techniques 
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and approaches through which they can achieve greater proficiency 
in the communication skills and thus utilize greater potential for 
social and economic adjustment. 



KEY FEATURES 

A multimedia laboratory approach is employed in order to initiate 
and maintain student interest and to provide the most efficient and 
effective medium for the presentation of each skill or concept. 
Because of the precision and control they afford, instrument 
techniques are used to provide instruction in perceptual and visual- 
functional skills, word recognition, and reading fluency. The in- 
herent interest provided by the many audiovisual approaches 
employed creates a highly motivating and stimulating learning 
atmosphere. 

Multimodal 1 '*• motion is used to compensate for the relatively 
limited education of the undereducated, underachieving student. 
Some students learn more effectively through an aural-oral ap- 
proach, some from a visual approach, some through kinesthetic 
activity, and others through an analytic approach. The use of a 
multimodal approach enables the student to capitalize on preferred 
learning styles and to develop greater proficiency in modalities in 
which he is less competent. 

The multilevel organization of Learning 100 allows a student to 
enter at his level of need and to progress at his own rate. The 
system consists of a “readiness” stage for non-readers and six 
graded levels of instruction. 

Non-readers enter the program at the RA level. Ten readiness 
sessions are devoted to building basic auditory and visual discrim- 
ination skills; eye-hand coordination; directionality; the abiln. to 
name, recognize, and copy numhers and the letters of the alphabet; 
the ability to pay attention and follow directions; a positive attitude 
toward learning; and other skills normally associated with the 
“readiness” period of reading instruction, A basic sight vocabulary 
of ninety words is established. 

Students reading at the equivalent of first-, second-, cr third- 
grade level enter the program at AA, BA, or CA respectively, At 
these levels, the major stress is on the acquisition of visual-func- 
tional and perceptual skills, and extensive sight vocabulary, inde- 
pendence in word attack through phonetic and structural analysis, 
and basic comprehension and study skills vital to both reading and 
listening. 

Students reading at the equivalent of fourth- through sixth-grade 
level enter the program at levels DA, EA, or FA respectively. At 
these levels students are assumed to have developed adequate 
Q ral and visual perceptual skills and some independence in word 
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attack. More attention is devoted to enlarging the sight vocabulary, 
to enriching word knowledge through awareness of multiple mean- 
ings, and to developing analytical, critical, appreciative, and selec- 
tive reading skills. In addition, instruction is provided in the ad- 
vanced reading comprehension skills, listening-auding skills, and 
study skills needed for mastery of content area information, 

Auto-instruct/onai techniques increase both involvement and 
learning time for each student by permitting him to respond to 
every question and exercise, and by furnishing immediate rein- 
forcement. Many of the materials and techniques are self-pacing so 
that each student can progress at a rate which is suitable for him. 

Personalized instruction is possible because the instructor has 
many opportunities to work with individuals or small groups during 
times when the majority of students are involved in independent 
learning. 

The adult-oriented content of Learning 100 was specifically de- 
veloped to meet the Leeds and interests of the target population, 

There are more than six hundred specially prepared reading and 
listening selections in three topic areas: 

1. Adjustment to Everyday Living 

These selections are designed to facilitate personal, social, 
and occupational adjustment. The majority are personal narra- 
tives, showing how various individurls met and coped with their 
problems. 

2. Living in Today’s World 

The selections in this group were prepared to fill gaps in the 
student's educational background. Information in the areas of eco- 
nomics, arithmetic, social studies, and science helps to provide a 
foundation for the attainment of elementary and secondary school 
equivalency. 

3. Enrichment Through Reading 

Here the student finds that learning can be entertaining as well 
as informative}. He reads or hears stories of adventure, suspense, 
and humor. He encounters some of the great literary classics, 
which add to his insight into human behavior and his understanding 
of the basic truths of life. 

Cycles of instruclion, which constitute the organizational pattern 
of the Learning 100 system, introduce and reinforce learning through 
a carefully planned sequence of activities. A cycle consists of four 
parts, each of which contributes to the development of specific 
skills, abilities, or concepts. Within each cycle, the student first 
receives perceptual accuracy and visual efficiency training. Next, 
he participates in activities which enrich his experiential background 
and prepare him for subsequent instruction. The third part of the 
cycle consists of a skill-building sequence which introduces and re- 
inforces vocabulary, word recognition, and comprehension skills, 




Finally, all the words, skills, and concepts taught during the skill- 
building sequence are applied by the student during independent 
reading activities in the fourth part of the instructional cycle. 

PART I— PERCEPTUAL ACCURACY AND VISUAL 
EFFICIENCY (5-10 minutes) 

Each cycle begins with a brisk five minutes 
of instrument training to develop high 
levels of accuracy and efficiency in the 
visual-functional and perceptual processes 
which initiate reading. 

PART II -BUILDING EXPERIENCES 
(15 — 20 minutes) 

A teacher-guided group discussion makes 
possible the establishment of a common 
experiential background which will serve 
as a framework for the instructional 
content to follow. 

PART III -SKILL BUILDING 
(90 minutes) 

Small groups of students move through a 
series of interrelated activities which 
assist them in the acquisition of new 
vocabulary, word attack skills, listening 
and reading comprehension capabilities and 
fluency in silent reading. 

PART IV -APPLICATION AND ENRICHMENT 
(20 — 30 minutes] 

The cycle culminates in a variety of 
activities in which the student applies 
the skills and vocabulary mastered during 
the skiU-building segment. 

Each instructional cycle can be completed in 2 V 2 to 3 hours. The 
RA cr Reaumess Stage consists of ten cycles which can be com- 
pleted in a minimum of 25 hours. Each subsequent level (AA 
through FA) consists of 30 cycles which require a minimum of 75 
hours. Thus, if groups met five days a week for 2 V 2 hours daily, 
any level beyond Readiness could be completed in 6 weeks. 

It is recognized, however, that some students will require more 
time to master skills and concepts. For such students, additional 
q Enforcement and a slower pace of instruction can be provided. 
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Continuous evaluation procedures are provided at frequent points 
during each cycle so that the instructor can quickly and easily 
identify those students who have mastered the skills and concepts 
necessary to continued progress and can detect those who need 
additional help before proceeding to the next segment of instruction. 

Guidance for instructors is provided in the comprehension manual 
of procedures and detailed lesson plans which are an integral part 
of the system, These provide all of the background information and 
daily * ‘dance needed to conduct Learning 100 classes. Explicit 
proceaui .-j coupled with the many auto-instructional activities make 
it possible for a teacher with little or no experience in reading in- 
struction or basic adult education to administer a Learning 100 
program successfully, 

Comprehensive skid development is provided through instruction 
in a broad array of communication skills judged to be essential for 
successful functioning in a technological society. As a result of 
careful analysis of the processes and purposes of reading and 
listening, the twelve areas listed below were isolated for attention. 
In addition, provision is made for instruction in the allied communi- 
cation skills of speaking and writing. 

The program is unique in the specificity with which the various 
communication skills and subskills are developed. Because most 
undereducated students lack proficiency in basic perceptual and 
visual-functional skills, intensive instruction in these skills is pro- 
vided, establishing a firm foundation for later instruction. The word 
recognition and comprehension skills developed at each level were 
selected on the basis of their importance to older students and are 
introduced in a developmental sequence. 

1. Perceptual Accuracy 

Identification, recognition, retention 

2. Visual Efficiency 

Binocular coordination, motility, directional attack 

3. Word Knowledge 

Word meanings in context 

4. Word Attack Skills 

Structural and phonetic analysis 

5. Comprehension Fundamentals 

Recalling, understanding, associating 

6. Interpretation 

Inferring, predicting, comparing, concluding, visualizing, 

sensing 

7. Analytical Reading and Listening 

Form, structure, ietail 

8. Critical Reading and Listening 

Relevancy, accuracy, validity, significance 
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9. Appreciation/ 

Sensitivity t interest, taste * y 

20. Refeience Skills A ^'i f 

Location of information, use of materials \ 

n 

11. Selective Reading * / , 

Skimming and scanning j i 

12. Reading and Study Habits / / 

Approaches, attitudes, interests, goals ^ 

INSTRUCTION AT THE RA (READINESS) LEVEL 

The Readiness program was designed for the student who has 
acquired few, if any, of the skills of literacy. 

He generally lacks skill in noticing small likenesses and differ- 
ences and in remembering visual forms. He therefore needs in- 
tensive training to develop visual discrimination and visual memory. 

Although he lives in a world filled with sound, he rarely listens 
with comprehension and hrs, in essence, “tuned himself out.” He 
thus needs to Jearn how to listen attentively and to remember what 
he hears. 

Illiterate students generally have poor eye-hand coordination; 
rrany even have trouble holding and guiding a pencil. They need 
specific instruction in forming letters and numbers as well as in 
recognizing and naming the numbers and letters. 

Left- to- right directionality is not an innate skill, but one which 
must be developed. The non-reader needs specific training in order 
to develop the left-to-right directional attack which is necessary 
for fluent, efficient reading. 

It i r vital that the student have the experience of reading mean- 
ingful material during the beginning days of class. Thus the estab- 
lishment of a sight vocabulary is begun during the very first cycle 
of instruction, and the student soon finds that he can read simple 
stories with understanding and enjoyment. 
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INSTRUCTION AT LEVELS AA-CA (1-3) 

The techniques and training procedures of Levels AA-CA build 
on the basic skills introduced and taught at the Readiness level. 

Continued attention is devoted to the expansion of sight vocabu- 
lary. Beginning on the AA level and continuing through the CA 
level, particular emphasis is given to developing independence in 
word attack through phonetic and structural analysis. Upon com- 
pletion of Level CA the student will have been sequentially intro- 
duced to an array of phonic and structural analysis skills which 
will enable him to independently unlock most of the words he will 
ercounter in day-to-day reacing. 
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Since context plays a role of prime importance in any ' 'xvity 
involving the use of words, students receive intensive instruction 
in how to use context clues in unlocking the meaning of words 
and in understanding both oral and written material. 

On Levels AA through CA, students are given instruction and 
practice in progressively more complex comprehension skills. The 
important skills of understanding main ideas and making inferences 
are introduced early in the AA level and reinforced in subsequent 
instruction. 

Other interpretive skills such as visualizing, predicting outcomes, 
and determining cause and effect are developed sequentially 
throughout these levels. 

It is also important that students become fluent and efficient 
readers. Beginning at the AA level and continuing through the CA 
level, several instrument techniques are used to develop skill in 
processing ideas and to build fluency in silent reading. 

During these levels students also receive developmental instruc- 
tion in those writing and speaking fundamentals which are practical 
and applicable to everyday living and working stations, 

INSTRUCTION AT LEVELS DA-FA (4-6) 

Students who enter Learning 100 at Level DA or above have as 
goals either entering the world of employment or completing ele- 
mentary equivalency and continuing their education. Therefore, the 
program is broadened to include a wider variety of communication 
skills instruction that is directly related to the immediate needs of 
these individuals. 

The instructional program at Levels DA-FA is organized into four 
parts, which require the same time intervals as the program at 
Levels RA-CA. 

At these levels, continued emphasis, dictated by student needs, 
is placed on developing high levels of accuracy and efficiency in 
the visual-functional and perceptual processes. There is also con- 
tinued emphasis given to the development of common experiential 
background and to the improvement of oral language facility. 

On the fourth, fifth, and sixth reading levels of Learning 100, a 
combined recording and workbook approach is used to introduce 
and give practice in a variety of listening, reading, and writing 
skills. 

At these levels, the Study Skills Library is used to help students 
develop the reading skills and approaches they need in order to 
read effeclively in the content areas. Lessons programmed for in- 
dependent learning provide step-by-step instruction in the areas of 
interpretation, evaluation, organization, and reference. 

During each cycle at these levels, students participate In teacher- 
guided activities designed to refine their ability to use context and 




to improve their spelling ability. Independent activities are provided 
which stress dictionary usage. 

As on the lower levels, continued stress is placed on developing 
fluent, efficient reading and improving the comprehension skills. 

LEARNING LABORATORY 

In essence, Learning 100 converts the classroom into a learning 
laboratory in which each student is encouraged to assume responsi- 
bility for his own daily activities and the progress he will make. 
The program is arranged so that the student is exposed to a var- 
iety of independent, small-group, and teacher-directed activities. 
Students move from one learning activity to another, completing 
prescribed cycles of instruction in an orderly, sequential manner. 

The carefully structured program of skill development places 
success within reach of students who have a history of repeated 
failure, and whose socioeconomic background has rendered them 
unable to cope with traditional curriculum. 
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THE RESEARCH PROCESS 




SELECT PROBLEM FOR RESEARCH 

Survey Literature to Review Rationale and Objectives of L-100 
System 

Although an educator undoubtedly had a general knowledge of 
-Jthe underlying theory of the L-100 system when he purchased it, 
discover that he needs to understand the theory more 
bcificaily in order to evaluate the system within his instructional 
^environment. Since L-100 is based on a specific purpose and ra* 
pnale (as stated in Chapter 2), the evaluator should begin by 
familiarizing himself with concepts on which this instructional sys- 
tem is based. These concepts include curriculum development for 
the educationally deprived, multimedia instruction, programmed 
materials, individualized instruction and perceptual and visual- 
functional skills. An examination of current literature should be 
reviewed at this time. Selected references can be found in the 
bibliography of this manual. 

Examine Learning 100 Syslem Materials and Procedures 

Next, it is vita! that one become thoroughly familiar with the 
nature of the system itself. This cen best be facilitated by first 
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reading the Instructor’s Manual, second, by reading and listening 
to and looking at selected lessons from various levels of the pro- 
gram and, lastly, by observing the system in action or by actually 
using the system with a group of students. 

The evaluation process grows out of the relationship between an 
educator's objectives and the instructional objectives of the educa- 
tional program. The educator, for example, may decide that a major 
objective of his program is to keep students motivated ?o that they 
attend classes. An additional objective may be that students be 
able to complete an eighm-grade or high school equivalency exam- 
ination. Passing such an examination is vital for the success of 
his program. 

Naturally an educator is primarily concerned with the extent to 
which the educational program fulfills his particular objectives. 
However, his objectives must be written operationally for a re- 
search project in terms of specific demonstration of achievement 
or performance. Objectives stated in terms c>f performance com- 
municate explicitly the kind of behavior the learner must exhibit 
as evidence that he has achieved the objective, the conditions 
under which this behavior is expected to occur, and the level of 
performance considered acceptable as evidence that the learning 
has taken place. 

To assist in planning an evaluation program, the objectives of 
L-100 are listed in Appendix A so that an educator may examine 
them in relation to his curricular objectives. The educator should 
compare his broad objectives with the L-100 overarching objectives, 
and then review the subsystem objectives which are written opera- 
tionally selecting those which he wants to use as a basis for his 
research project. One must be cautious, however, when selecting 
operational objectives for a study, In evaluating an Instructional 
program, the educator must be certain that he isolates the objec- 
tives which he wants to examine, holding all factors constant in 
his evaluation except those which he plans to evaluate. In evalu- 
ating two instructional programs, for example, LrlOO and another 
set of instructional materials, one must be certain that all factors 
in the study other than the two instructional programs remain 
constant, 

FORMULATE RESEARCH QUESTIONS 

After the concepts, materials, and procedures of the L-100 sys- 
tem have been carefully examined and the educator’s general in- 
structional objectives determined and compared to the objectives 
oi the system, the next step would be to formulate questions based 
on an understanding of the theory and application of L-100 and a 
knowledge of the particular setting in which L-100 is being used, 
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Such questions might include: 

1. Will a systems approach be beneficial within my instructional 
environment? 

a. What will be the comparative gain of students using a sys- 
tems approach compared to students using a conventional 
approach? 

b. Will students react more favorably to the systems approach 
than to a conventional approach? 

2. Will a multimedia system provide for an increase in student 
motivation and learning? 

a. To what extent do the multimedia techniques influence the 
attendance patterns of adults or young adults? 

b. Will exposure to a multimedia system encourage greater 
classroom interaction? 

3. Will L-100, an individualized learning system, provide students 
with an efficient learning experience? 

a. Will students using Learning 100, an individualized learning 
system, make greater gains than those using a less indi- 
vidualized approach? 

b. In terms of age, day or evening students, urban or rural 
settings, what groups will demonstrate greater gains in 
reading achievement? 

4. What will be the reactions of teachers who use the Learning 
100 system in their classrooms? 

Formulating questions is not an easy process. If a question is to 
be useful for evaluation, it must be answerable by available means 
of observation or experimentation in the existing enviionment. The 
question should also make clear the nature of the observations that 
have to be made in order to find an answer. Often there is a 
tendency to ask questions which cannot be answered readily by 
evaluation. For example, one might ask the question, “What effect 
does exposure to the Learning 100 system have on the self-concept 
of the students?" This type of information is valuable but it is 
more difficult to measure objectively with available instruments. 
The teacher may observe an improvement in the seif-concept of 
her students but attributing it to the use of the instructional system 
is often difficult to prove. However, every attempt should be made 
to ascertain what changes occur in the affective domain. 

After an educator has decided what questions he will explore, 
these questions become the basis for fcirm'iting hypotheses. A 
hypothesis is a tentative assumption or a sluiement of the problem 
or problems of the study in testable terms. It may be stated either 
as a null hypothesis, the "hypothesis of no difference," or as a 
research hypothesis which can be accepted if the null hypothesis 
can be rejected. 
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The null hypothesis is expressec when the purpose of the study 
is to determine whether or not a difference exists between groups. 
A statistical test is employed which will enable the educator to 
determine if the differences between the groups are significant or 
if they can be attributed to chance. The research hypothesis, how- 
ever, is a statement of expected outcomes based on theory or 
previous research. It suggests that in a certain specified situation a 
particular outcome would be expected. The hypothesis states the 
outcome and suggests the kind of experiment that might be de- 
signed to test the hypotnesis. 

For example, an evaluator looking at some of the questions men- 
tioned previously might develop certain hypotheses for evaluation. 
Stated in the null form, example hypotheses might be: "There will 
be no difference between standard scores or the Stanford Achieve- 
ment Test earned by experimental (L-100) ar d control (conventional 
program) group students.” "There will bn no difference in the 
frequency of attendance of students using Learning 100 and stu- 
dents using a conventional reading program. " "There will be no 
difference in the reactions toward instructional materials by ex- 
perimental group teachers using L-100 end the control group 
teachers using traditional materials, as determined by a teacher 
questionnaire." 

The corresponding research hypotheses might be: "Scores earned 
(on the Stanford Achievement Test) by the experimental group 
using L-l^o will be higher than scores earned by the control group 
using a conventional program." "The frequency of attendance Tor 
students using Learning 100 will bo higher than for students using 
a conventional reading program as determined by attendance 
records," "Teachers using L-100 will react more favorably toward 
L-100 materials than will control group teachers toward conven- 
tional materials, as determined by a teacher questionnaire." 

It is important for an evaluator to develop a hypothesis which is 
suitable for experimentation. In A Guide to Educational Research, 
Robert M. W. Travers has suggesled the following characteristics 
of good hypotheses: 

1. They should be rooted in a framework of theory or previous 
research. 

2. They should be testable. One reason why hypotheses may be 
untestable is the possibility Ilia 1 one or more variables cannot 
be measured for some reason. Another general source of un- 
testable hypotheses are area* where little or no organized 
knowledge exists on which lo formulate clear and testable 
hypotheses. 

3. Iney should state relationships between variables. 
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4. They should be limited in scope. It is better to get a clearly 
defined corner of the univers- in your hypothesis than to try 
something of global significance which may in reality require 
a lifetime of work 

5. They should be consistent with most known facts. 

6. They should be stated, as far as nossib’.a, in simple terms. 
Simplicity can increase clarity at no loss of significance. One 
aspect of simplicity is avoiding vague terminology, 

7. Finally, hypotheses should be amenable to testing in a reason- 
able length of time. (Cook, editor, pages 181-182) 

Good hypotheses are based on asking good research questions. 
However, not all p.ublems can be stated in the form of a hy- 
pothesis. For example, in a descriptive survey where one is de- 
scribing an existing situation, there would be neither a statement of 
expected outcomes as in a research hypothesis, nor any statistical 
test of differences between groups. In a descriptive survey one 
might formulate a question such as, "What are the various means 
of grouping employed by teachers using L-10J in the classroom?" 
A study of this type could serve as background for a future ex- 
perimental study by suggesting areas where more formal research 
is desirable. 

PREPARE RESEARCH DESIGN 

Having stated a hypothesis or hypotheses, the educator then de- 
velops a specific research design. A design is a method for con- 
trolling the variables during the course of an experiment. Although 
many types of designs may be developed, three types are of par- 
ticular relevance to the educator as he attempts to evaluate instruc- 
tional materials. 

The first is the "new program versus the traditional." This type 
may be employed to seek information regarding the effectiveness 
of one program ol system over another. Information gained would 
serve as a basis for determining whether more materials of a par- 
ticular type should be purchased and for determining groups for 
which the materials are most successful. 

The second type is the "program A versus adaptation of pro- 
gram A" design. An educator faced with scheduling problems at 
his school because or an increasing student population may wish 
to experiment wish several variations in scheduling to determine 
how he can best adapt the system to suit his needs. He would 
then compare the achievement test scores of the two or Ihree 
groups to determine the effect of the variation on the progress 
of each group. An educator may wish to determine the effect of 
scheduling the lessons in one continuous block of lime contrasted 
to the same amount of time spread over «vo days. 
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The third type of design is “program A versus program A under 
different conditions. 11 With this design, the educator might want to 
determine whether the system is more effective with one particular 
group of junior high or senior high students than with another. 
Both groups of students would be exposed to L-100 under similar 
conditions. The outcome of this type of study would provide in- 
formation which would enable the educator to decide where to 
concentrate his efforts in the future. 

At this point, an educator who is familiar with the local prob- 
lems may wish to call in a research consultant who is familiar 
with research practices to help in the selection of an appropriate 
plan of action for conducting the evaluation. The consultant can 
offer guidance to facilitate the collection of appropriate data which 
will be used to analyze the objectives being tested. 

SELECT SAMPLE GROUP FROM POPULATION 

Selection of a sample from the population for a study is also an 
important consideration. Population refers to the total number of 
individuals constituting a given group of persons or things that 
have some common observable characteristics. In selecting a sam- 
ple, the educator should examine the setting in which the instruc- 
tional program exists. He should also be aware of the environ- 
mental conditions from which the students come. Setting is con- 
cerned with the characteristics which help to make a particular 
school or institution unique and may indicate some of the problems 
inherent in each. An educator should objectively consider his setting 
in terms of the following factors: 

1. Socioeconomic status 

2. Regionalism 

3. Degree of urbanization 

4. Physical characteristics 

5. Philosophical charac- 
teristics — innovation 
oriented 

6. Size of district 

7. Size of classes 

8. The education and prior 
experience of teachers 
(certified or para- 
professional) 

9. Type of school or^7N£T ^ 
tut>on (community center; 
job corps, secondary 

L% school) 
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The characteristics of the students in a Learning 100 class, adult 
or secondary, need to be considered when selecting a sample 
for study. The following are some of the factors which may be 
examined: 



1. Age 

2. Sex 

3. Race 

4. Language spoken at home 

5. Years of formal schooling 

6. Geographic area where 
majority of schooling was 
obtained 



7. Previous school achieve- 
ment 

8. Scholastic aptitude test 
scores (if available) 

9. Attendance record 

10. Year of most recent 
employment (ABE only) 



Conditions within classrooms also need to be as similar as pos- 
sible. Some of the questions which the investigator should ask in 
considering experimental and control classes to be used in the 
study are: 

1. What time of day do the classes meet? 

2. Do the classes meet for the same length of time each day? 

3. How much time is devoted to reading instruction and/or 
communication skills instruction? 

4. Are the classes taught by certified teachers, para-professionals 
or others? 

5. What are the background characteristics of the teachers which 
may have some effect on the students within the classes, i. e. , 
lengih of service, educational level, and experience with 
grouping? 

6. How much training have the teachers received in the use of 
the instructional materials being employed in the study? 

7. Is out-of-class preparation time provided for the teachers? 

When selecting a sample for the study, the evaluator may decide 

to use an intact class of students, such as a classroom group within 
a secondary school. When this is the case, the evaluator may wish 
to locate another similar class within the same school district or 
area which can serve es a control class: or he may want to use 
data compiled in other years for previous classes, at the same 
grade level and with the same teacher, as a control. However, in 
selecting a sample of intact classes, subtle t’as or additional vari- 
ables often are introduced. Therefore, the researcher must try to 
isolate in advance as manj of the variables as possible and small 
initial differences between experimental and control groups can be 
statistically adjusted. 

Random selection of a sample will eliminate or "neutralize" 
some of the differences which would occur within intact group3. 
If the total population were grouped together and some students 
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were randomly assigned >o the experimental class and the rest to 
each of the control classes, the researcher could confidently believe 
he had eliminated systematic bias and had reduced the effects of 
individual variations, The theory underlying random sampling im* 
plies that the larger the sample, the less bias exists. Thus, if the 
sample is large enough, one would expect to find in the population 
about the same number of low-ability students, about the same 
proportion of males and females, and about the same proportion 
of persons of a given age. 

Another similar technique which can be used in selecting a 
sample is called blocking. Similar students are grouped together 
and the students in this group are then randomly assigned to qn 
experimental or control class. For example, if all males wei^; 
grouped together and then randomly assigned to one group or 
another, one could later evaluate the effects of the system on a 
similarly assigned group of females. This procedure could then be 
repeated with an age range as a variable so that in comparing the 
experimental and control groups for each subgroup, differences due 
to sex and age will have been eliminated. 

DETERMINE METHODS FOR COLLECTING DATA 
TO EVALUATE OBJECTIVES 

The next step in developing the research design is to decide 
what types of evaluation instruments to use to collect data for the 
research project. Many evaluation techniques are available although./^— 
some of these are less well known. To assist you in choosinfl 
appropriate evaluation techniques from the many available ones;! 
Appendix B of this manual presents a taxonomy showing types 
techniques that may be used for evaluation. 

In considering the sample research hypotheses stated previously 
one finds that these hypotheses can be evaluated in a number 
ways other than those indicated below. 

1. "Scores earned (on the Stanford Achievement Test) by the 
experimental group using L-100 will be higher than scores 
earned by the control group using a conventional program." 

2. "The rate of attendance for students using L-100 will be 
higher than for students using a conventional reading program 
as determined by attendance records," 

3. "Teachers using L-100 will react more favorably toward L-100 
materials than will control group teachers toward traditional 
materials, as determined by a teacher questionnaire." 

Although standardized test ,, teacher questionnaires, and use of 
teachers 1 logs are valid techniques commonly employed, other 
techniques may also be suitable. Local or teacher-made tests may 
be utilized rather than standardized tests as a measure of gains. 
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Observational techniques may take the place of or be used in 
adjunct with teacher questionnaires. As an indication of motivation, 
feacher anecdotal records may be used as well as attendance 
records. 

The evaluator must consider his objectives and the objectives of 
the instructional program when selecting the evaluation instrument 
to be used. A standardized test which does not reflect what is 
being taught to the students is of little value. Similarly, a teacher 
questionnaire which does not ask questions pertinent to the ob- 
jectives of the evaluation is merely a waste of time. 

For the evaluator concerned with adult basic education, suit- 
ability of tests for adult basic education classes is a problem. In 
many cases, evaluators have had to rely on standardized achieve- 
ment tests developed for children. Some, therefore, have estab- 
lished their own norms for the tests using their own population or 
similar populations as a basis or they have adapted available tests 
for local use. Recently, however, tests designed for use by adults 
have been marketed. One may refer to sources such as those in- 
cluded on page 61 of the bibliography to gain the most up-to-date 
information on various tests. 

If a researcher decides to use subjective evaluations such as a 
questionnaire or teacher observation techniques, he may find that 
he will need assistance in preparing the instruments. References 
are included in the bibliography and samples of questions for a 
questionnaire can be found in the Appendix. 

Another concern when determining methods for collecting data is 
that the evaluator consider all data that might be necessary for his 
study. It is not mandatory that he use all the data he co^ects but 
if he has neglected to gather some vital information his study may 
not be as complete as he had hoped. For example, an evaluator 
may decide that he wants to keep a record of why students drop 
out of L-100 to ascertain whether it is related to the instructional 
program or to personal factors. 

He may also wish to consider attendance as well as dropout 
rate. If students attend only half of the class sessions, their ex- 
posure to the materials would certainly be limited. This could be 
a contaminating factor in the research study and should, therefore, 
be part of the record. 

The evaluator may also be concerned with what happens during 
the class periods. For this purpose he might ask the instructors 
to keep logs or anecdotal records of their daily classroom activities 
as well as records of daily progress by students and their reactions 
to the materials. In addition to the daily progress reports of instru- 
ment speeds during the various forms o r training as the student 
progresses through the cycles, the teacher may wish to keep an 
accounting of the number of supplementary books each student 



reads and book reports and other papers completed. From these 
daily records, the evaluator might extract information that would 
aid him in the analysis of how students progress through the 
program. 



DETERMINE METHOD OF ANALYZING DATA 



For an evaluator to determine whether the hypotheses of his 
study should be accepted or rejected, it is necessary that he em- 
ploy a method to analyze the data. Analysis of data includes a 
comparison of the outcomes of the various instructional approaches 
used with the groups being examined and the making of a decision 
as to the achievement of the goals of the research. One of the 
more common methods of analysis used for evaluation is statistics. 
Statistics are numerical facts, but statistics is also a body of meth- 
ods to aid in making decisions when dealing with only a sample 
of the entire population over a relatively brief period of time. 
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There are many aspects to be considered in selecting statistical 
methods. A prime consideration is appropriateness of the technique 
to the situation in terms of the evaluator’s desired outcomes since 
different statistical techniques provide slightly different information. 

Descriptive statistics are used to describe data by summarizing 
sets of numerical data, such as test scores or years of education, 
indicating measure of central tendency and variability. The mean, 
or arithmetic average is the most commonly used measure of 
central tendency. This indicates where the scores appear to center 
or group together on the scale used. Measures of variability reflect 
the spread in a set of scores. 

Inferential statistics concerns itself with the relationship between 
variables and the inferences that can be drawn from the observed 
small sample to a larger population. Statistical techniques can be 



E? 





27 



utilized to determine the degree of confidence t h it can be placed 
in inferences about such sample relationships, A statistical test 
attempts to determine the probability that chance alone underlies 
an observed relationship, for instance that test score is related to 
reading approach. 

Specific statistics are used depending on the type of scale of 
measurement one obtains by the quantification process. Quanti- 
fication is the process of assigning numerical value to data that 
normally would be qualitative. Data are quantified in four basic 
ways, commonly referred to as the nominal, ordinal, interval, and 
ratio scales. For explanations of these scales, refer to the glossary 
of this manual. 

Research in education usually makes use of the nominal, ordinal, 
and interval scales while the ratio scale is used primarily in the 
physical sciences. Two general types of statistics may be used to 
analyze data measured by these scales. Non-parametric tests, sta- 
tistical tests that do not depend upon the assumption of normality 
in the distribution of sample or population scores, are most useful 
with the first two types of measurements and, to some extent, 
with the third. Parametric tests, statistical tests based on the as- 
sumption that the scores were drawn from a normally distributed 
population, are most useful with the interval scale level of measure- 
ments. (Parametrics are less useful with ordinal and nominal scales.) 

The decision as to which statistical technique to use may not be 
one which the evaluator can make on his own unless he is familiar 
with rtatistical methods. He may wish to review Appendix C, 
Outline of Statistical Tests, which briefly describes some of the 
statistical techniques available to the educator-evaluator. In addition, 
he may wish to consult with statisticians and experts in the area 
of educational measurement in order to be able to select an appro- 
priate technique with which he can work comfortably. 

There are many books of varying degrees of complexity dealing 
with statislical analysis, references to some of which are in the 
bibliography of this manual. Skimming through several books nv'iy 
serve to refresh the thinking of the evaluator who has had prior 
experience with statistics. And, for the inexperienced researcher, 
they may be able to provide background information and definitions 
which will enable him to deal more adequately with the assistance 
a statistical consultant can provide. 

PREPARE FLOW CHART 

After determining the objectives of the study, the sample, and 
the instruments to be used for evaluation, the next step is to de- 
velop a realistic timetable. A flow chart, such as the example on 
page 30 , outlines all the procedures which must be undertaken in 
a study and the time by which each procedure must be completed 
if the ensuing activities are to be carried out within a specified time. 
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Whenever any one of the activities listed on the chart is not com- 
pleted on time, each of the subsequent activities will probably be 
delayed. 

By preparing a flow chart of research activities, the educator- 
investigator can plan schedules so that students and teachers will 
be interrupted as little as possible. This procedure also aids in 
determining the cost of a research project. If one realizes that the 
data must be computed at a particular stage which falls within a 
specific budgetary period, the costs can be provided for in advance. 
Also, by charting activities one can work around scheduled holi- 
days and other known alterations in the regular schedule. 

If a flow chart is prepared carefully and includes all the major, 
steps necessary for evaluation, it can be revised should any un- 
expected problems arise and still retain the continuity of the re- 
search plan. Problems such as the following might arise: 

1. A serious epidemic may occur. 

2. Weather conditions may be unusually bad. 

3. Teachers scheduled to teach may not fulfill their obligations 

for various reasons. H 

4. Budgetary provisions may be lowered suddenly. ^ 

5. A fire or other emergency may destroy the teaching facilities.' 

6. The person in charge may neglect to administer tests at a 
particular time. 

7. Research consultants may not be available at a specified time. 

8. Delays may occur because of mail problems. 

9. Computer time may not be available when desired. 

Any one of these problems could force the evaluator to make 
alternate provisions for the research project. Naturally, he would 
want to salvage as much of the research as possible. This is one 
reason it is important to forsee potential problems and be prepared 
for alternate actions. 

The evaluator may find, for example, that the time specified for 
teacher interviews is not feasible because there is a flu epidemic 
in the community. After serious consideration, the evaluator may 
decide that he will alter the design and have teacher interviews at 
a later date. At the same time, he should realize that the informa- 
tion may be different. For example, teachers may respond differ- 
ently after working with L-100 for a period of time. However, the 
data can still provide valuable feedback if the change is taken into 
consideration. 

If alternate provisions are to be made, however, the evaluator 
should examine his flow chart carefully and revise his timetable so 
that adequate time is allowed for each subsequent stage in the 
research. 

The accompanying flow chart shows the schedule of activities for 
a sample L-100 research project. 
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FLOW CHART OF A SAMPLE RESEARCH PROJECT 



Activity Steps 



June July August September October November 



1. Involve members of the 
total community to ensure 
cooperation. 

2. Review theory by search 
through literature, examine 
L-100 system. 

3. Determine broad objectives 
and subsystem objectives 
which specify desired out- 
comes. 

4. Determine hypotheses to be 
tested. 

5. Determine type of research 
design. 

6. Select sample from popula- 
tion. 

7. Determine method for evalu- 
ating objectives; if necessary, 
create instrumentation. 

8. Order evaluation instruments. 

9. Teacher orientation. 

10. Administer pretests. 

11. L-100 instruction. 

12. Distribute and collect ques- 
tionnaires, surveys. 

13. Observe learning situation. 

14. Administer posttests. 

15. Analyze data. 

1G. Interpret data. 

17. Report results. 

18. Revise curriculum or continue 
evaluation. 

o 
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ADMINISTER THE STUDY 




Those trying to institute change may meet with some public 
resistance to their efforts. As many members of the community as 
is possible should be involved in the evaluation program, such as 
parents when secondary students are to be participants, interested 
lay individuals or groups, and professional personnel of the schools. 
Also administrative officials, politicians, community leaders, and 
citizens within a community should be informed of the purposes of 
evaluation. Often the costs involved in research seem unreasonably 
high to the layman who does not understand what research entails. 
Here the evaluator may use community group gatherings, the mass 
media, and opinion leaders to aid in communicating the purposes 
and procedures of research to the various publics he serves. 

When secondary students are involved, the parents may have to 
be prepared for the fact that their children may receive different 
treatment from that provided for other students. Questionnaires 
may also have to be sent home which will enable the evaluator to 
make socioeconomic classifications or to accumulate other back- 
ground information and it is important for parents to feel that their 
privacy is not being invaded. By early involvement of the parents 
in the study, it is hoped that undue pressures which may affect the 
results will be eliminated. 

Respect for privacy is equally important when adults are the 
students. They may be very sensitive about divulging information 
regarding their attitudes and personal lives through interviews and 
questionnaires. It is important that they understand that the collec- 
tion of this type of information is an integral aspect of research 
but in no way will be related to them personally. ID numbers may 



be used instead of names on all data 
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instruments. 
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A principal should be made aware of the evaluation design so he 
can offer comments and suggest alternatives before the procedure 
is actually begun. If he knows what will be expected of him and 
of his staff, he may be inclined to assume certain responsibilities 
or m;»k i administrative arrangements to facilitate the evaluation 
efforts. He may permit an unusual scheduling procedure such as 
double periods or provide particular facilities such as a learning 
laboratory. 

Many teachers are concerned that their students do well com- 
pared with others, that their students are at least average, and that 
people think well of them. It rests with the researcher to acquaint 
those involved with the purpose of the project which is to evaluate 
a program, not an individual, a teacher or a school system. Teach- 
ers must be made to feel that they are in a cooperative venture 
and that research is not being forced upon them. Educational inno- 
vation may be threatening to teachers, and the possibility of evalua- 
tion along with innovation may cause some of them to adopt a 
negative attitude toward both the curriculum and the evaluation 
procedures themselves. The teachers who will be working with the 
Learning 100 materials should be oriented so that they conceive of 
their role as 'hat of consultants. They must be made aware of the 
fact that they are not being singled out for evaluation but are 
assisting in ,» total evaluation of the instructional program. Control 
teachers must also be included in training and planning sessions and 
made aware of the importance of their role in the research. 

In addition to the problems of communicating the purpose, value, 
and results of research to the persons concerned, the evaluator 
may also have problems in administering his design. 

To avoid possible contamination of data, the investigator should 
try to be aware of; 

1. Outside influences that might be responsible for contaminating 
the measures of gain or loss 

2. Inaccuracies in collecting, recording, analyzing or reporting 
data 

3. Standardized tests not administered exactly as directed or ad- 
ministered at inappropriate omes 

4. Introduction by the teacher ot supplementary materials not 
provided for in the study 

5. The teacher who is not sufficiently familiar with the instruc- 
tional program and who uses instruments and materials in a 
manner other than that recommended by the producer 

An important factor to keep in mind when dairying out the 
research is to attempt to control the teacher variable. Unless one 
is trying to evaluate the influence of the teacher, one must attempt 
to keep the teacher variable as constant as possible, that is, try to 



assign teachers who have similar backgrounds and training to the 
classes involved in the research. Within many adult basic education 
classes there is great diversity in their training and backgrounds, 
Some have had little 01 no training in reading instruction, Oihers 
have had little experience with the disadvantaged, particularly with 
disadvantaged adults. Some are certified teachers. Others have 
gained all their training or experience in the classroom as para- 
professionals, One should also gather as much of this background 
material as possible so that it can be considered in light of the 
results of the study. Also, the amount and type of teacher training 
in use of innovations such as L-100 programs should be reported. 

A twofold problem exists in recognizing and anticipating problems 
which may occur with materials. First, there is the problem of 
having the necessary instructional materials which are being used 
in the study. The evaluator should ensure that enough of the ap- 
propriate materials have been obtained to provide for students who 
may enter the classes at various times during ihe term. Often, 
because of budgetary problems, there is a tendency to share in- 
struments and equipment in such a way that prohibits the instru- 
ments or materials from being available as required in the program. 
Beyond this, conserving materials by not using workbooks consum* 
ably, often decreases the effectiveness of many of the lessons and 
increases the difficulty of completing these lessons. 

In addition, the use of instruments may be a departure for many 
teachers. In the orientations, enough time should be provided for 
the teacher to become fully comfortable with the operation and 
maintenance of the instrument:!. 

The second part of the problem has to do with the evaluation 
materials, These include tests, questionnaire forms, interview sched- 
ules, end other such items. They should be ordered or prepared 
well enough in advance so that they are available to the researcher 
at the time specified. Many research projects have been delayed 
merely because the tests did not get to the classroom in time. 

The problem of mobility of students is one to be considered. If 
the school conducting the research is in an area where transfers 
are high, provisions should be made to select a large enough sample 
to compensate for this loss. 

Another threat to the validity of the study is the ‘’Hawthorne 
Effect,” a special motivation effect that can change the behavior 
of the experimental teachers and pupils, Attempts should be made 
to compensate for the special treatment this group is receiving. 
Possibly a placebo situation can be set up in which the control 
group is made to feel as important as the experimental group by 
offering some additional materials that have not previously been 
used in their classroom. If the study is a comparison of the effec- 
tiveness of a traditional or rxisting program with that of L-100, 
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care must be taken to ensure that this placebo treatment is essen- 
tially an extension or an integral part of the existing program 
rather than a special treatment itself. Another possibility is the 
designation of two control grc ,ps, one using a conventional pro- 
gram, the other an alternate innovation. 

An investigator's sensitivity to what might happen places him in 
a better position to meet the various contingencies and sharpens 
his awareness of what is occuring during the transactions. 



ANALYZE DATA AND INTERPRET RESULTS 



After the data from the study have been collected and prepared 
for analysis, the evaluator must analyze the data and interpret the 
results. Depending on the size of the study and the research funds 
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available, the data may be analyzed manually, by ijse of an adding 
machine, or by use of a computer. Sometimes a combination of^|he 
methods may be employed. For example, questionnaires may bfc 
tabulated manually while the test results may be analyzed by a 
computer. 

As was indicated previously in this manual, a statistical consul- 
tant will be of considerable help in selecting the appropriate sta- 
tistics for the research and in processing the data. He will also be 
able to explain the statistical findings in objective terms. 

However, the interpretation of the findings in relation to this 
instructional situation must always be made by the educator. He 
may find one group made statistically significant gains over another 
group on the achievement test, but the group that did less well on 
the achievement test significantly increased their listening skills. 
Therefore, the educator would need to determine what was re- 
sponsible for this discrepancy in relation to the entire study. He 
would then report the findings and his interpretation of the findings 
based on the data received from the study. 



REPORT THE RESULTS 



( / 

(j'J •"'< / 

?- 7 — 

/ ' / fifc'SuLr j 

)X4 ? : — 

i \ vi • . , 

tkvM 

\Y\ nr"X“: 

Mf rJY 



The final research procedure is to describe the study, either 
formally in a written account or informally, perhaps before com- 
munity or parent groups, other teachers, or the Board of Education. 
The simplicity or degree of sophistication of the written report will 
depend upon the audience for whom it is intended. There is no 
need to go into explanations of complicated statistical procedures 
for a non-research-oriented audience. A report that summarizes the 
objectives and findings in nontechnical terminology may be the 
best approach. 

Translating a research study into a written report can have great 
value if it is presented clearly, comprehensively, and accurately. 
This report may be referred to repeatedly by the school district or 
the findings may be passed on to others via publication in a pro- 
fessional journal or in addresses at local or national conventions. 
Replications of the study as well as additional related research 
studies and longitudinal investigations then become possible. 

Before an attempt is made to write the report, the making of a 
detailed outline based upon the research design created at the onset 
of the project will help to organize the material into a logical 
sequence. The time it takes to draft a good outline will in the long 
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run p*nve an economy. In spite of their differences, research 
studies are similar enough to suggest a typical organization such as 
the following: 

I. Presentation of the Problem With Which the Study Deals 

A. Summary of local issues 

B. Statement of solution being sought 

II. Background for Study 

A. Related literature 

B. Prior studies 

III. Procedures Employed 

A. Description of population and samples u/>ed 

B. Objectives or hypotheses being tested 

C. Instrumentation and data collection methods 

D. Summary tables 

E. Methods of analyzing data 

1. Statistical tests 

2. Levels of decision 

IV. Results 

A. Presentation of findings 

B. Aralyses of findings 

V. Discussions 

A. Reference to objectives or hypotheses being tested 

B. Implications and recommendations for additional studies 

VI. References 

A. Bibliography 

B. Appendix, if desired 
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The first section dealing with the presentation of the problem 
might include background information explaining the importance of 
the particular study including a summary of local issues, a state- 
ment of the purpose of the study and its significance, hypotheses 
and limitations of the study, underlying assumptions, and definitions 
of important terms. 

The section dealing with related literature Indicates the pertinent 
knowledge gained through a review of the literature on the problem 
and summarizes other research studies relevant to the present 
investigation. 

Next, there is usually an account of the procedures employed, 
describing how the design has been carried out. Here, information 
is presented regarding the location of the study; the size and 
selection of the sample; the incomes, occupations, and educational 
levels of students (ABE) or the families of the students included in 
the sample; the experience and educational background of the 
teachers involved; and experimental and control techniques. The 
instruments selected and the reasons for their selection, methods of 
data collection, and time schedules should be described. A clear, 
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understandable statement of the methods of analyzing the data, 
including statistical tests used and levels of significance decided 
upon, is needed. 

The section dealing with results and conclusions of the study is 
a crucial one. The value of the research lies in the consistency of 
the conclusions drawn from the results obtained. One approach that 
may prevent inconsistency is to organize the chapter by restating 
each hypothesis tested, to give the related data in tables and text, 
and then to formulate a conclusion. If this format proves too 
repetitive, results for several hypotheses might be summarized in a 
single table permitting meaningful comparisons among related data. 
Generous use of tables enables the reader to study and compare 
numerical data efficiently. 

The section on results and conclusions is also an appropriate 
place in the written report to present a summary of the research 
performed, A short review at this point refreshes the reader's 
memory without requiring that a profusion of details be read. A 
brief statement of the problem area, the specific purpose of the 
investigation, testable hypotheses, the methods and procedures 
used, the procedures for obtaining data, and some of the more 
important results will aid the reader to refocus on various aspects 
of the total research problem. 

The section discussing the implications of the study should also 
refer back to the broad objectives being tested and outline the 
findings in relation to the problem being investigated. In addition, 

'the relationship of results of past research and the present findings 
might be discussed. Ideas on extended meanings of the findings for 
additional studies may be stated here. Negative, nonsignificant, or 
unexpected findings should be included. An identification of the 
problems not solved by the present research might suggest recom- 
mendations for future research. Possibly there are faults in the 
study which a critical analysis of the design will bring to light. 

IMPLEMENT FINDINGS IN THE CURRICULUM 

After an educator has completed the evaluation of a particular 
instructional system within his educational setting, he then exam- 
ines the findings in order to make decisions regarding curricular 
change. Perhaps the most central, but often the least used function 
of evaluation is to validate the hypotheses upon which the curric- 
ulum is based. When establishing an evaluation procedure such as 
that described in this manual, it is first necessary to review instruc- 
tional objectives and to examine them in relation to the objectives 
of the particular innovation that is to be used. As a result of the 
study, an educator may determine that his instructional objectives 
need to be revised, or he may find that the innovative program 
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can justifiably replace the existing program or that adaptations of 
the innovative program will be necessary to produce optimum 
results in his particular situation. The forward-looking educator, 
however, keeps in mind that all curriculum plans and approaches 
to instruction are only hypotheses which need to be tested. This is 
true of established curriculum practices as well as innovations. 

Another important function of the evaluation of instructional 
programs is to provide information on student achievement. The 
very continuity of curriculum and learning depends on such infor- 
mation. Many problems of articulation between the various levels 
of schooling exist because of insufficient knowledge about what 
precisely the students have mastered on the previous level. 

The innovative school is the school of tomorrow — not the school 
of yesterday which relies primarily on precedent in its approach to 
education. The atmosphere for innovation should be initiated from 
within a school, not imposed from without. The school adminis- 
trator who is challenged by change will encourage teachers who 
might otherwise be reluctant to enter into the mainstream of 
innovation. 

A necessary adjunct to innovation is the research process. This 
process, too, must be carried out in an atmosphere of communica- 
tion and understanding among the faculty, students and public. 

Research associated with innovation should be an ongoing pro- 
cess. However, research need not suggest a vast undertaking as 
small studies may be profitably used to document trends as a part 
of the overall process. 

It is hoped that this manual will provide encouragement to utilize 
the school or classroom as a setting for research associated with 
educational innovation. This research, which can be of significant 
practical value, can shape the future of education within the school 
of tomorrow. 
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OVERARCHING AND SUBSYSTEM 
OBJECTIVES OF LEARNING 100 



OVERARCHINC OBJECTIVES 



• To develop perceptual accuracy and visual efficiency 

• To enrich experiential background 

• To develop an extensive vocabulary 

• To develop word attack skills 

• To develop and increase ability to direct and sustain attention 

• To develop fundamentals of reading and listening comprehension 

• To develop interpretative reading and listening skills 

• To develop critical reading and listening skills 

• To develop analytical reading and listening skills 

• To develop the ability to locate and use reference sources 

• To develop efficient study skills and learning approaches 

• To encourage individuals to progress at their own rate and to interact directly 
with the materials in an independent manner 

• To provide a continuous and effective program of student evaluation and rein- 
forcement 

• To provide the instructor with an instructional system that will interest and moti- 
vate undereducated adults, out-of-school youths, and potential dropouts to develop 
basic communication skills 



LEVELS RA-CA 

Perceptual Accuracy and Visual Efficiency 

• To develop ability to recognize rapidly, retain accurately and in a left-to-right 
direction symbols, numerals, and letters presented by the Tach-X 

• To develop a sense of spatial relationships as it relates to reading and writing; 
accurate visual discrimination as it relates to the identification and recognition of 
numerals and letters; and facility in writing numerals and letters by use of a 
workbook 

• To develop ocular facility and strengthen habits of good directional attack, as 
they relate to silent reading, by scanning elements projected in a left-to-right 
manner on the Controlled Reader at rates that approximate fluent silent reading 

Building Experiences 

• To prepare the students for subsequent instruction by voluntary discussion and 
student response to questions through instructor-mediated activities 

• To encourage ease of student interaction and improve oral language facility by the 
relating of experiences through instructor-mediated activities 

• To encourage development of social living skills 
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Skill Building 
Word Introduction 

• To build a vocabulary using an Aud-X presentation 

• To develop independence in word attack skills within the context of a word 
study lesson using an Aud-X presentation 

• To reinforce a recognition vocabulary within the context of an Aud-X presenta- 
tion involving independent activity 

• To develop a variety of comprehension skills through listening experiences pro- 
vided by the Aud-X 

Word Recognition Training 

• To develop sight recognition of all words introduced by the Aud-X by reducing 
the recognition time through the use of the Tach-X 

• To develop ability to use context clues as an aid to word recognition through 
use of workbook activities 

• To introduce elementary principles of language usage that concentrate on word 
changing principles through instructor-mediated activities 

• To further reduce recognition time of sight words and increase the rapidity with 
which an individual associates words by presentation of a running context on 
the Controlled Reader 

• To extend comprehension skills by student response to questions relating to a 
running context presented on the Controlled Reader 

Reading Fluency Development 

• To reinforce and integrate all the visual, functional, perceptual and associative 
skills acquired in the preceding skill-building activities by Controlled Reader- 
mediated silent reading exercises 

• To further develop comprehension skills through independent Controlled Reading 
workbook activities 

Application and Enrichment 

• To further develop sight recognition of vocabulary by means of the Flash-X 

• To reinforce the perceptual and associative skills acquired in the preceding skill- 
building activities by means of student and instructor-mediated workbook exercises 

• To develop and reinforce skills of comprehension, interpretation, and evaluation 
by means of workbook and instructor-mediated activities 

LEVELS DA-FA 

Perceptual Accuracy and Visual Efficiency 

• To develop ability to recognize rapidly, retain accurately and in a left-to-right 
direction symbols, numerals, and letters presented by the Tach-X (as needed) 

• To develop ocular facility and strengthen habits of good directional attack, as they 
relate to silent reading, by scanning elements projected in a left-to-right manner on 
the Controlled Reader at rates that approximate fluent silent reading (as needed) 




Building Experiences 

• To prepare the students for subsequent instruction by voluntary discussion and 

student response to questions through instructor-mediated activities 

• To encourage ease of student interaction and improve oral language facility by the 

relating of experiences through instructor-mediated activities 

• To encourage development of social living skills 

Skill Building 

Listening and Study Skill Development 

• To develop listening comprehension skills (Level DA), listening comprehension 
and reading skills (Level EA) f and listening comprehension and writing skills 
(Level FA], through the use of recordings or tapes and independent workbook 
activities 

• To develop reading skills of interpretation, organization, evaluation, and refer- 
ence by student-mediated manipulation of study skills materials 

Word Recognition and Spelling 

• To develop ability to use context clues as an aid to word recognition by use of 
workbook activities 

• To develop sight recognition of words introduced by reducing the recognition 
time through use of the Tach-X or the Flash-X 

• To develop spelling skills through use of the Tach-X and a workbook and in- 
structor-mediated activities 

• To develop skills in dictionary usage through workbook activities 

Reading Fluency Development 

• To reinforce and integrate all the visual, functional, perceptual and associative 
skills acquired in the preceding skill-building activities by Controlled Reader- 
mediated silent reading exercises 

• To further develop comprehension skills through independent Controlled Reading 
workbook activities 

• To reinforce the perceptual and associative skills acquired in the preceding 
skill-building activities by means of student and instructor-mediated workbook 
exercises 

• To develop and reinforce skills of comprehension, interpretation, and evaluation 
by means of workbook and instructor-mediated activities 

• To further develop and reinforce word attack skills by means of the Aud-X and 
independent workbook activities 

• To reinforce ability to utilize comprehension skills by means of the Controlled 
Reader and independent workbook activities 
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•Taxonomy developed by The Institute for Educational Development. New York. N.Y., John Kenmdy. Vice President 
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APPENDIX C 

OUTLINE OF STATISTICAL TESTS 



Experimental design and statistics can be the life work of a specialist but it can 
also be a useful tool for tl j teacher, the administrator, and the members of the 
school board. It can answer questions with a predetermined degree of precision that 
we would otherwise have to answer with subjective judgments or opinions such as, 
"I feel this curriculum will be good for the kids,” or, ”1 hope this curriculum will 
be good for the kids.” 

There are complex formulas and computer and talk of split-plot and Latin 
Square designs, but tliese are also basic designs that are appropriate and useful and 
for which the logical rationale should be clear to a reader regardless of his degree 
of statistical sophistication. 

A few such experimental designs will be introduced in this section which have 
been selected because they can be used in most school-oriented research. The 
assumptions and the formulas will be left in the adequate hands of the textbook 
authors who have been cited in the bibliography. The concern here is only of sta- 
tistics as a tool for decision making and improvement of the individual learning 
situation and environment. 

The t-test 

In many experimental situations one measurement and only two groups, the ex- 
perimental and the control group, are required. A classroom might be used as a 
population and students would be randomly assigned to one group or the other, or 
a school system might constitute the population and classroom units would be as- 
signed to treatment groups. This design would be useful for testing the relative 
effectiveness of a traditional and an experimental curricula. For example, when only 
two groups are involved, a statistical test, the t-test, is appropriate. The t-test value 
reflects the degree of difference between two means and the value obtained from 
computation of the t-test formula can be checked against prepared and readily 
available tables to judge whether or not the computed value from the data is sig- 
nificantly large enough to be considered statistically different from chance. A brief 
look at the computational formula for the t-test and a little simple arithmetic will 
show that three factors affect the size of the computed value. 

1. The magnitude of the differences between means of the two groups should be 
large, the larger the difference the better the chance for significance. 

2. The amount of variance, differences within each group, is also vital. If the 
groups have large variability a great deal of overlap is present, but if the 
variance of the groups is relatively small, far less difference between means 
is required to give significant results. 

3. Finally, the she of the sample [the total number of individuals or classroom 
units) is important. As the sample size increases, the value that you computed 
from the t-test formula increases and the experiment is more apt to yield sig- 
nificant results. 

In summary, the t-test is an extremely useful and fairly simple way to test the 
significance of mean difference between two instructional approaches. 

Analysis of Variance 

If three or more groups are involved in an experiment, the t-test is no longer an 



appropriate test but an extention of the t- test ( the F-ratio can be used. The F-ratio 
is the arithmetic end result of a statistical test called Analysis of Variance. When 
testing the degree of difference between means for three or more groups the vari- 
ance within the individual groups is used.. It seems circular and most confusing but 
a simple diagram suggested by Popham(1967] makes the whole process seem quite 
logical. 

Assume a classroom teacher or an administrator wished to test a traditional pro- 
gram against two different experimental programs. In other words, he has a control 
group and two experimental groups. After he administers the posttest to the three 
groups he can graph his results horizontally* using the same scale for all groups and 
the total sample. Figure 1 shows this graphic representation. 




Mean X = 103 
Control Group (C) 




Mean X= 104 

Experimental Group I (E^ ] 
(Treatment I) 




Experimental Group II ( E 2 ) 
(Treatment II) 



/ 





C + E 1 + E 2 = Composite graph 



FIGURE 1 
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The so-called null hypothesis, the statement that no difference exists following 
administration of the treatments, is certainly true in this case. It is easy to see that 
when the results for all the subjects are graphed together as in the bottom graph in 
Figure 1, the variances or spread of scores is little different from the variance with- 
in the individual groups. 

Now assume that the teacher or administrator wishes to test two other experi- 
mental programs. Following administration of the experimental treatments he again 
graphs the results of the posttests. Figure 2 shows these results graphically. 




Control Group (C) 




(Treatment 11) 





FIGURE 2 

At 



In this case, the null hypothesis cannot be considered to be true. It is obvious 
that when the three groups are graphed together as in the bottom graph in Figure 
2, the variance or spread of scores is far greater than the variance of a *y single 
group. 

It is possible to test to see if any two of these groups, for example Treatment I 
and Treatment II t are significantly different from each other. This is done by 
multiple-comparison tests, and is well defined by Winer (1962), It is best to procure 
the assistance of a research design consultant in interpreting the results of a multiple- 
comparison test. This technique could be very useful to an administrator faced with 
the decision of which new materials should be purchased. If Treatment I is not 
significantly different from Treatment II and yet Treatment II is only sixty per cent 
as costly as Treatment I, the school board would be much happier to purchase 
Treatment II for the district. 

In summary, the Analysis of Variance and the resulting F-ratio is a design which 
can be used to advantage when three or more groups are involved in the experi- 
ment. It is really an extension of the t-test and, therefore, the factor of size of 
differences between means, size of variance within the groups, and sample size have 
much the same effect. 

Analysis of Covariance 

A favorite word of statisticians is power. They talk of powerful tests and how to 
achieve power. It means, quite simply, that if one goes to the trouble to run an 
experiment, as much of the chance bias and experimental contamination as possible 
must be removed. One way to increase power is to remove by arithmetic proce- 
dures as much of the variance as possible that is due to factors other than the 
treatments administered to the subjects. Remember that a small variance is desirable 
if we hope to get significant results. 

Even if the groups are selected at random, as statisticians demand, it is very dif- 
ficult to select groups at random that aie truly equal on all dimensions such as 
spatial ability, auditory discrimination, and paragraph meaning. The variance due to 
these initial and almost inevitable differences can be removed by using a statistical 
formula that makes the groups statistically equal. The formula is a combination of 
Analysis of Variance and regression analysis. The formula appears forbidding and 
the arithmetic does become complex, but a design specialist and a computer can 
unravel the signs and symbols very quickly. 

The Analysis of Covariance technique can be used appropriately and to great 
advantage with two or more groups when initial and final measurements are avail- 
able and offers us a powerful statistical tool. It is a bit more complicated arith- 
metically, but the results may be far more gratifying. 

Factorial Design 

To this point, only experimental designs that test one dimension have been dis- 
cussed, for example, testing the relative effectiveness of treatments in order to make 
a decision about different curricula. Now assume that the curricula with respect to 
different kinds of students is to be tested. The evaluator might want to know if 
Treatment 1 is more effective for high-ability students and Treatment II is more 
effective for low-ability students. Even though Treatment 12 costs less money, it 
might be possible to buy Treatment I for certain areas or types of students if it 
proves to be particularly effective. 
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Now a design lhat is two or more dimensions is needed. To the one dimension 
treatment discussed before, a second dimension, ability level, is added. The design 
schemata would then look like this: 




High Avg. Low 
ABILITY LEVEL 
FIGURE 3 



Each experimental unit is now placed in the appropriate cell. For example, the 
scores of a high-ability student who was randomly placed in the control group 
would be in the upper left cell and the score of an average-ability student randomly 
placed in Experimental Group 1 would be In the middle cell. 

Now, again with some arithmetic, one can test the curriculum as well as gain 
information to help to make decisions about the programs for different types of 
students. The statistical concept of interaction rears its complex head at this point 
and design consultants will be happy to draw graphs and talk of four-factor inter- 
actions which have little interpretable meaning for our curricular improvement. Two- 
factor interactions, however, are not only meaningful but very helpful. 

Correlation 

Often an evaluator wants to know whether things go together or seem to be at 
odds with each other. For example, it would be helpful to know whether or not all 
children who attend school every day consistently get the best grades in the class. 
If this is true, a correlation coefficient can be computed and a high value expected 
(less than but very close to -F1.0). This would indicate that the work done in the 
classroom by those students with high attendance was very eifective. If, however, 
the computed correlational value was low (very near zero and either positive or 
negative) one would be led to believe that school attendance had relatively little 
effect on grades or achievement in school. Now assume that the computed correla- 
tion coefficient is high but negative (greater than but very close to —1.0). This 
would mean that the children who attend school regularly are actually being hurt by 
the experience and students with poor attendance are becoming better achievers. 

The correlation technique is simple, though tedious to use, and requires only 
measurement of the two factors of interest. After the correlation Is computed, a 
test is available in textbooks that can be used (or one can refer to a table in many 
statistical books) to determine whether the correlation is really significant or simply 
due to chance. Smaller values of the correlation coefficient are significant if the 
sample size is large enough since a large sample will provide a more representative 
index of the relationships under consideration. 

Avery useful tool, the correlation coefficient can provide information relationships 
such as age and vocabulary, parents’ occupational level and achievement scores, and 
attendance and curriculum program effectiveness. 

Use the correlation coefficient generously. It ran provides great deal of meaning- 
ful information to the teacher and evaluate! 

so 



APPENDIX D 

SAMPLE QUESTIONNAIRES 

LEARNING 100 END OF YEAR REPORT 



Teacher Date 

Location Installation Number 



List alphabetically the names of the students in your class, last name first. 

Indicate for each student: 

1. The last cycle and level completed. 

2. Exposure speed at Tach-X for Tach-X Word Recognition exercises for the last cycle and level com- 
pleted. (Indicate exposure speed as A, B, C, D, E, or F.) 

3. Projection rate for Controlled Reader Story for the last cycle completed. (Indicate rate as 20, 30, 40, 
50, 60, 70, 80, 90, 100, 110, 120, or 130.) 

4. Projection rate for Processing Training for the last cycle and level completed. (Indicate rate as 20, 30, 
40, 50, 60, 70, 80, 90, 100, 110, 120, or 130.) 





U) 


(2) 


(3) 


(4) 




Last Cycle 




Controlled 


Processing* 




and Level 


Tach-X 


Reader 


Training 


Student’s Name 


Completed 


Dial Setting 


Dial Setting 


Dial Setting 
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cate P for Processing Motor. R for Regular Motor 
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Check one appropriate statement in each of the following groups. 



Students’ interest level in the use of the instruments: 

Remained high during the school year 

Started high and diminished as time progressed 

Started low and increased as time progressed 

Remained low during the school year 



Teacher reaction to the program: 

I am comfortable with the program and enjoy using it 

I am comfortable with the program but do not enjoy using it 

I feel overwhelmed by the program and do not enjoy using it 

I feel overwhelmed by the program but still enjoy using it 



Teacher evaluation of the program: 

The program is superior to others 1 have taught 

The program is comparable to others I have taught 

The program is inferior to others 1 have taught 

This is my first teaching experience 



Check as many statements as apply; 

Too much time is involved in planning for each day 

One teacher can easily handle the program 

I would like to use this program again next year 

Substitute teachers can manage the program if the teacher is absent 



Have you had any assistant working with you in the L-100 program this year? 

Yes No 



If yes, indicate below the type of assistant you have had. 

How Many? How Frequently? 

Teacher’s Aide ___ 

Student Teacher 

Certified Teacher 

Reading Specialist 



List number of students in each reading group (Aud*X or Controlled Reader) in your classroom: 
Group 1 Group 2 Group 3 Group 4 Group 5 _ 
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When scheduling problems arise and you must deal with the pressures of too little time to complete all 
the suggested L-100 activities in the time available, which of the L-100 activities listed do you eliminate 
occasionally? 



Please check as many as necessary. 





1. Building of experiences for Aud-X stories 


— 


2. Planning of class and personal agenda for the day 




3. Tach-X ABC Accuracy Training 




4. Controlled Reader Motility Training 




5. Aud-X Story and Word Study (Levels RA-CA) 




6. Aud-X Listen, Listen and Read, Listen and Write (Levels DA-FA) 




7. Tach-X Word Recognition Training 




8. Controlled Reader Processing Training 




9. Controlled Reader Fluency Training (stories) 




10. Check gains in reading rate and comprehension with Reading Efficiency Checks 




11. Planning reading activities 




12. Conducting teacher-pupil conferences 




13. Using Study Skills Library 




14. Using Flash-X to practice words 




15. Reviewing Aud-X, Tach-X or Controlled Reading lessons according to specific needs 




16. Practice handvvritir g 




17. Independent reading or directed reading in GO 




18. Sharing of If .ary books 




19. Reading favorite stories, or poems orally 




20. Dieting or reading into tape recorder and listening to recording 




21. Class or group discussions of topics of interest 




22. Practical and creative writing 




23. Review word attack skills in Aud-X Word Attack Review Study Guide 




24. Directed reading with Comprehension Power filmstrips 



Approximately how much time each day is devoted to: 

Perceptual Accuracy and Visual Efficiency Training minutes 

Building Experiences minutes 

Skill Building minutes 

Application and Enrichment — minutes 

Total time devoted minutes 



Place a check in the column that best describes your opinion of the instructional effectiveness of each of 
the L-100 materials listed below: 



Outstanding Good Average Fair Poor 



Instructor’s Manual 
Cycle Lesson Plans 
Tach-X Accuracy Filmstrips 
Eye-Hand Coordination Workbook 

Controlled Reading Accelerated Discrimination Filmstrips 

Motility Training Filmstrips 

Aud-X Word Introduction Filmstrips 

Aud-X Study Guides 

Tach-X Filmstrips 

Tach-X Word Recognition Books 

Controlled Reading Study Guides 

Controlled Reader Filmstrips 

GO Books 

Processing Filmstrips 
Study Skills Library 
Reading Efficiency Checks 
Aud-X Word Attack Review Book 
Comprehension Power Filmstrips 
Listen Lesson Book 
Listen and Read Lesson Book 
Listen and Write Lesson Book 
Flash-X Word Discs 





CONTROL CLASS QUESTIONNAIRE 
END OF YEAR REPORT 



Teacher Date 

Location. Installation Number 



1. What instructional program are you using with your class? 



2. List supplementary reading or skill materials used in your classroom in addition to the basic instructional 
program. 



3. Approximately how much time each day is devoted to: 

Skill Building 

Application and Enrichment . 

Additional Activities 

4. Have you had any assistant working with you this year? Yes No 

If yes, indicate below the type of assistant you had. 

How Many? How Frequently? 

Teacher’s Aide 

Student Teacher - 

Certified Teacher 

Reading Specialist 

5. List the number of students in each reading group in your classroom. 

y Group 1 Group 2 Group 3 Group 4 . — . Group 5 
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GLOSSARY OF RESEARCH AND EVALUATION TERMS 

USED IN THIS MANUAL 



Autoinstructional Devices — Systems and instru- 
ments for individual instruction, including in- 
dividual reading devices, individual viewing and 
listening equipment, language laboratories, and 
programmed printed materials 

Control Group — In an experiment, the group or 
groups that are not subjected to the experi- 
mental factor, and with which the experimental 
group is compared 

Data — Factual material used as a basis especially 
for discussion or decision 

Data Analysis — The interpretation of numbers, 
facts, or other quantities used to draw conclu- 
sions or to make statistical inferences 

Evaluation — The process of judging the value of 
something by careful appraisal, informally by 
casual observation and subjective judgments, or 
formally by controlled comparisons 

Experiment — A tentative procedure or policy ac- 
companied by control of conditions and/or con- 
trolled variation of conditions together with 
observation of results to test or establish a 
hypothesis 

Experimental Group — In an experiment, the group 
that is subjected to the factor being tested in a 
study of its affect on the group 

Hypothesis — A conjectural statement of the rela- 
tion between two or more variables 

Instrument: Instructional — An electrical or me- 
chanical device used to implement an educa- 
tional program 

Instrument: Testing— A measuring device for de- 
termining the present value of a quantity under 
observation 

Interval Scale — A fcrm of measurement where 
tem3 are selected so that the scale intervals 
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between the items are equal; having all the 
characteristics and relationships of the ordinal 
scale, except that the distances between any 
two numbers cn the scale are of known and 
equal size,- the ratio of any two intervals is 
independent of both the zero point and the unit 
used. (In other words, the zero point and the 
unit of measurement used on the scale are 
arbitrary. Thus, an IQ of 120 does not indicate 
twice as much intelligence as an IQ of 60 since 
there is no such quantity as zero IQ) 

Mean — The sum of a set of scores divided by the 
number of scores; commonly called the average. 

Measurement — The practice of testing, scaling, 
and appraising aspects of the educational pro- 
cess and of the individuals undergoing the edu- 
cational process 

Multimedia Approach — Instruction that utilizes a 
variety of media: filmstrips, records, tape re- 
cordings, books, instruments, etc. 

Multimodal Approach — Instruction that enables 
children to capitalize on their preferred learning 
style: aural-oral, visual-oral, tactile stimulation, 
etc. 

Nominal Scale — A form of measurement which 
classifies items into two or more categories 
without any reference to their magnitude (such 
as identifying "boys" and "girls” as B and G 
or as 1 and 2) and with no particular order 
assigned to the categories, allhough numbers 
may be assigned to them 

Norm — Average or typical performance 

Objective — Something toward which effort is di- 
rected; an aim or end of action, a goal 

Operational Objective— A description of what the 
learner will be doing, the conditions under 
which the behavior will occur, and the criterion 
of acceptable performance 

Sb 
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Ordinal Scale — A form of measurement used 
when the categories within a scale not only 
differ but also have a distinct relationship such 
as a preferred order allowing the researcher to 
assign values that permit observations to be 
placed in relative rank order only; no implication 
as to the distances between positions can be 
assigned to the ranking; it assigns observations 
to categories by number and arranges them in 
some logical order (such as ranking achievement 
on a particular test instrument from high to low] 

Overarching Goals — Long-range general state- 
ments which reflect the anticipated outcomes of 
the program in terms of the author’s philosophy 
rather than in terms of siudents' demonstrable 
behavior 

Population— A total group of individuals, objects, 
or items from which samples are taken for 
statistical measurement 



Probability — The likelihood that any one of a total 
number of events will occur in a situation in- 
volving chance 



Program — A number of activities properly orga- 
nized into learning units for the purpose of 
attaining specified educational objectives 

Programmed Instruction— The utilization of pro- 
grammed materials to achieve educational objec- 
tives; synonymous with autoinstruction, auto- 
mated teaching, etc. 

Random Sample — A sample drawn in surn a way 
that every member of the population has an 
equal chance of being included, thus elimincting 
bias of selection, it is “representative” of its 
total population 



Ratio Scale — A form of measurement using equal 
units throughout the range of the scale, an abso- 
lute zero point representing complete absence of 
the property being measured and the ratio of the 
scale units independent of the unit of measure- 
ment. (For example, six pounds are twice three 
pounds, six feet are twice three feel, and six 
minutes are twice three minutes.] All quantifica- 
tion and computation which can be performed 
by the nominal, ordinal, and interval scales may 
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also be accomplished on the ratio scale and also 
it is possible to divide by the categories or di- 
vide by the numbers on the scale itself 

Reliability — The degree to which an evaluation 
instrument measures accurately or consistently 
whatever it proposes to measure 

Research Design — A plan by which research sam- 
ples may be selected from a population and 
under which experimental treatments are admin- 
istered and controlled so that their effect upon 
the sample may be measured 

Sample Population — The individuals selected from 
a total population, or universe, for the purpose 
of studying certain characteristics that will re- 
veal truths concerning the population from 
which they are selected 

Search (seeking material) — The operation to de- 
termine whether certain information is available, 
the manner in which it is organized, and where 
it is located 

Skill — An ability that is essential to successful 
performance 

Statistically Significant — Having a high probabil- 
ity, as shown by statistical procedures, of being 
due to the operation of factors other than 
chance 

Subsystem Goals — The goals, relating to the sub- 
systems of a system, which are descriptive and 
qualitative in nature 

System — An operational configuration involving 
interlocking and interdependent instructional 
techniques and materials presented in a func- 
tionally related learning environment in order to 
achieve specific educational objectives 

Taxonomy — Orderly classification 

Validity — The accuracy with which any statistic, 
test, or other measuring device measures what 
it purports to measure; the extent to which the 
scores obtained from a test or measuring scale 
can be used for prediction 
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EVALUATION PLANNING GUIDE 




1. Select problem for research 



a. Survey literature to review rationale and objectives of L-100 system 



b. Examine L-100 materials and procedures 



18 



c. Examine instructional objectives of educational system 
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2. Formulate research questions 
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Determine hypotheses to be tested 



3. Prepare research design 



4. Select sample group from population 



5. Determine methods for collecting data to evaluate objectives 



6. Determine method for analyzing data 



7. Prepare Flow Chart 



8. Administer the study 



9. Analyze data and interpret results 



10. Report the results 



11. Implement findings in curriculum 
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